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A statistical analysis of runway incursion (RI) events was conducted to ascertain 
relevance to the top ten challenges of the National Aeronautics and Space Administration 
Aviation Safety Program (AvSP). The information contained in the RI database was found 
to contain data that may be relevant to several of the AvSP top ten challenges. When 
combined with other data from the FAA documenting air traffic volume from calendar year 
2000 through 2011, the structure of a predictive model emerges that can be used to forecast 
the frequency of RI events at various airports for various classes of aircraft and under 
various environmental conditions. 


Nomenclature 


ANOVA = Analysis of Variance 

ASIAS = FAA Safety Information Analysis and Sharing 
ATADS = Air Traffic Activity Data System 
AvSP = NASA Aviation Safety Program 
Cl = Confidence Interval 
CY = Calendar Year 

DX8 = Design-Exert software (Version 8) from Stat-Ease, Inc. 

FAA = Federal Aviation Administration 

LSD = Least Significant Difference bars 

METAR = A format for reporting weather information 

NASA = National Aeronautics and Space Administration 

RI = Runway Incursion 

RS = ResponseSurface 


I. Introduction 

O NE focus areaofthe National Aeronautics andSpace Administration (NASA), enabled through the Aviation 
Safety Program (AvSP) of the NASA Aeronautics Re search Mission Directorate, is to improve aviation safety. 
The AvSP 1 (http://www.aeronautics.nasa.gov/programs_avsafe.htm) seeks to provide increasing capabilities to: 

• predict and prevent safety issues 

• monitor for safety issues in-flight and lessen their impact should they occur 

• analyze and design safety issues out of complexsystembehaviors, and 

• analyze designs and operational data for potential hazards 


Within this domain, the is sue of runway safety is one thrust of investigation and research. One component of the 
runway safety thrust is that of runway incursion (RI) events. Runway incursions, as defined by the Office of 
Runway Safety 2 oftheFederalAviationAdministrationAviation (FAA), are the incorrect presence of an aircraft, 
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vehicle or person on theprotected area of a surface designated for the landing and take-off of aircraft, as reported by 
the respective air traffic control personnel. 

Looking into the literature on this topic, a recent NASA study on non-towered airports’ indicated that the number 
of RI events is increasing with time, with about half of the events being of low severity and the remainder being spl 
among moderate, high, and severe RI events; among these events, intersecting runaways are noted as the highest 
contributing factor. A recent presentation bythe BoeingCompany 4 shows that flighthours, departures and the size 
of the worldwide fleet have generally increased, while accident rates have remained essentially flat (but at a very 
low level) overthe last 20years ; the same presentation points to about 6% of all accidents being associated with 
final approach, landing, takeoff and initialclimb. A recent U.S. Department of Transportation, Volpe Center 5 report 
shows that the spacing of parallel runways has just a small effect (if any) on the number of RI events across all 
severity categories; the same reports illustrates that crossing the hold short line, entering the runway and crossing a 
runway as the most likely types of RI events. A recentjoumal article 6 illustrates a dramatic increase in the number 
of RI reported in 2008 compared to previous years, with pilot deviations always being the largest source of these 
events. A recentFAA report 7 described the strong correlation among airport geometry, complexity and various 
communication tools (including signage and runway markings) with RI events. A Pilots As sociation report* 
illustrates the increase in RI events with air traffic, but with overall the RI event being less than 6 per million 
operations; this reports also points to major domes tic airports (Chicago, Atlanta, Dallas/Fort Worth, Los Angeles, St. 
Louis and Philadelphia) as having the greatest number of RI events. 

To that end of improving runway safety, a statistical analysis of the Runway Incursion (RI) Database 9 from the 
FA A Safety Information Analysis and Sharing (ASIAS) website was conducted to ascertainits relevance to the top 
ten challenges of AvSP. The information contained in the RI database was found to contain data that may be 
relevant to several of the AvSP top ten challenges 1 including: 1) the assurance of flight critical systems [i.e., airport 
operations], (2) the discovery of precursors to safety issues, and 3) improve crew decision-making and response in 
complex situations. 

When combined with other data fromthe FA A, documenting air traffic volume from calendar years 2000 through 
2011, the initial structure of a predictive model emerges thatis used to forecastthe frequency of RI events at various 
airports and under various environmental conditions. 


II. Methodology 

The scope of the workdetailed in this paper employs two commercially -distributed software products: Microsoft 
Excel and Design-Expert (version 8, referred to herein as DX8) from Stat -Ease, Inc 10 . The general workflow that 
was employed in this study was to first to download the RI data set fromthe ASIAS web site. Then, the air traffic 
volume data set" was downloaded fromthe FAA Data & Research / Aviation Data & Statistics / Air - Traffic 
Activity Data Sy stem(ATADS) / Airport Operations web site. These datasets were downloaded in Microsoft Excel 
format and this software was used to sort and extract the information of interest in addition to some statistical 
processing. The intent of this data pre-processing was to develop representative marginal and conditional 
probabilities of specific events, causes, combinations of contributing factors, and the participant types (aircraft 
classes, and if vehicles or pedestrians were involved) of RI events that occurred. In this context, it is not necessary 
that these searches andsortsbe 100% accurate, but merely that they provide reasonable guidance about the relative 
percentages. Having prepared the data set into suitable formats, the data was then imported into DX8 for the 
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development of response surface (RS) models via the analysis of variance (ANOVA) technique, and for additional 
statistical processing with the software 12 . 

The softwarechoices noted above simply represent software currently available to the author, and software 
packages to which the author is quite familiar, but in no way represent an official federal government or NASA 
endorsement of these software packages. However, these software packages are known to include the desired 
capabilities for accomplishing the objectives of this study. 

The R1 database consists of 10459 records for RI events from October 1, 2001 through September 30, 2011. 
Obviously, only data from part of the calendar year (CY) is included in this set for 2001 and 2011. Hence, 
information for CY 2001 was extrapolated from the existing CY 2001 data by using a multiplicative factor of 4; 
likewise information forCY 201 1 was extrapolated by using a multiplicative factor of 4/3. Each record includes 
the following fields : Event ID, Event Local Date, Event Local Time, Event State, RI Category, Airport ID, Event 
Location, Takeoff/Landing Runway, Aircraft 1 Type, Aircraft 2 Type, Aircraft 1 FAR category, Aircraft 2 FAR 
category, Weather Condition, and a Narrative Summary. RI Category, noted above, is a qualitative measure of the 
levelof risk associated with each event. In orderto enable numerical processing of this field, an assumed numerical 
risk value was associated with each qualitative category identifier. The categories employed within the RI database 
include: 


• A / Collision - an actual collision between two objects occurred (Assumed Risk = 5) 

• A - a collision was narrowly avoided (Assumed Risk = 4) 

• B - significant potential for collision existed (Assumed Risk = 3) 

• C - a RI event occurred with ample time and/or distance to avoid a collision (Assumed Risk = 2) 

• D - A runway incursion with no immediate safety consequences (Assumed Risk= 1) 

• Other (E, N/A, P, Assumed Risk =0) 


The Aircraft -type fields identify the manufacturer and modelof the primary (and secondary, if present) aircraft 
involved in the RI event, if applicable. The FAR Category fields classify the primary and secondary object(s) 
involved in the RI event, including aircraft, ground vehicles, and pedestrians, if applicable. The primary FAR 
aircraft categories are 121 (commercial), 135(air taxi) and 9 1 (general aviation), but there are other categories such 
as MAINT (taxi of an airplane by a non-pilot/ mechanic when the aircraft needs maintenance or when it needs to be 
moved from one parking position to another), MIL (military aircraft), PED / VEH (the secondary object was a 
pedestrian or ground vehicle, respectively), and other aircraft of less interest to this study, such as FAR categories 
125 (business aircraft), and 129 (foreign aircraft). 

The air traffic volume data set provided quantitative measures of how many landings and takeoff (grouped 
together) occurred by year at each of over 400domes tic airports in several categories of aircraft, including itinerant 
air carriers, air taxis, general aviation and military aircraft, and local civil and military aircraft. The total air traffic 
volume is for each airport is also provided. These datasets were used to investigate issues suchas the percentage of 
the air traffic volume (total, or at a specific airport) that resulted in runway incursions over a given period of time, 
and the true, traffic -normalized risk level is associated with those RI events. Again, the intent of the various data 
analysis operations was to develop representative marginal and conditional probabilities of specific events, causes, 
combinations of contributing factors, and participant the types of RI events that occurred. The data operations need 
only provide reasonable guidance about the relative percentages. 
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III. Results 


The number of RI events fromtheRI database was analyzed as function of calendar year (CY), Figure 1, and was 
found to be increasing overtime in a statistically significant way. The term “statistically significant” takes on a 
specific meaning in this context. A “statistically significant” change is the smallest change is the response of 
statistical significance to a chosen confidence level, here 90%, as indicated by the Least Significant Difference 
(LSD) bars enclosing the linear RS predictive model. The number of RI events reported for each calendar year is 
shown (red dots) as a functionof calendar year. A linear RS predictive modelfromDX8 is also shown in the figure, 
along with the 90% confidence interval (Cl) LSD bars. Hence, the number of projected RI events for 2012 is 
predicted to be about 1289, but any number ofRI events between about 1205 and 1372 would be “the same” (as in 
not distinguishable) froma statistical significance consideration. Furthermore, the number of projected RI events 
are increasing as functionof calendar year in a statistically significant way, since the LSD bars for CY=2001 and 
CY=201 1 do not overlap. 

As discussed above, relativeto Field 5 in the RI database, an assumed risklevelwas associated with each of the 
RI events, based upon their severity category, as assignedby the FA A Office of Run way Safety. The assumed risk 
value s umfor reported RI events fromthe RI database was analyzed as function of calendar year and found to be 
increasingas function ofcalendaryearin a statistically significant way, as shown in Figure 2. Also, a significant 
increase in the assumed risk sum value occurredoverthe period2007 and 2008, compared to the previous risk sum 
levels. This may be attributed to an RI definition change by the FA A between 2007 and 2008, described in a 2010 
paperby Chapman 13 . The same reference also presents several key findings of importance to the runway safety 
domain, namely: 

1. RI at non-towered airports are not reported in the data set analyzed 

2. More RI events have been classified as lower risk in 2010 than similar events in 2002 and 2003 

3. Pilots consistently rated RI events at higher severity than FAA 


The first of these findings is troubling since more airports may utilize non-towered control in the future due to the 
potential of budget cuts. The second finding is also troubling in that it may suggest a trend toward downgrading of 
the severity of RI events over time. Lastly and especially troubling in that it also reflects the difference of 
perspective from a pilot involved real time in a crossing paths event, versus an outside observer (air traffic 
controller) looking down on the scene. 

The air traffic volume (combined landings and takeoffs) data set fromthe FAA Data & Research/ Aviation Data 
& Statistics / Air Traffic Activity Data System (ATADS) / Airport Operations web site was analyzed for several 
different classes of aircraft as a function of calendar year from 2000 through 2011. Despite many documents 
showing trends with increased numbers of passengers overtime (e.g., Ref 4), there is a statistically significant 
decrease in the itinerant air earner traffic volume (combined landings and takeoffs) as afunctionofcalendaryear, as 
shown in Figure 3. As many air travelers today would attest, planes are more likely to be at full capacity today than 
in previous years. Likewise, there is a statistically significant decrease in the itinerant air taxi, general traffic and 
military air traffic volume as a function of calendar year. 

The RI database includes entries from 492 domestic airports; a unique tracking number was assigned to each 
individual airport to facilitate analysis and presentation. As shown in Figure 4, the number of reported RI events 
varies considerably fromairport to airport. Nearly half (48%) of all the airports considered had 10 or fewer reported 
RI events. About two thirds of the airports considered had a number of reported RI events below (in a statistically 
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significant way)themean value (about 21) of all the airports considered. A few airports, such as INT (Winston / 
Salem, NC) and FW A (Fort W ayne, IN) had a number of RI events well above the mean value for all the airports 
considered (228 and 164, respectively). 

As previously stated, the assumed average risksummation canbeanalyzed as a function of the airport number. 
This is shown in Figure 5. In this case, airports with large traffic volume, such as ORD (O’Hare, Chicago, IL) and 
ATL(Atlanta,GA) clearly standout with statistically significant highaveragerisksums. Again, many airports are 
found to be below the mean value (about 17) of all airports considered. When the average risksummation is 
normalized by the air traffic volume associated with each airport, othersmaller airports stand out with statistically 
significant highrisklevels. This indicates thateventhoughmany RI events occur at ORD or ATL, because of the 
large traffic volume at these sites, a flyer’s actual risk of being involved in a RI event at one of these high traffic 
volume airports may actually be significantly lower than the risk at other smaller volume airports. 

The data previously considered and reported as RI event count by year is now analyzed as a function of risk 
severity category andyearto determine which liskseverity categories are increasing overtime. As shown in Figure 
6, the only liskseverity categories thathaveexhibited a statistically significant growth over time are those for risk 
severity categories C and D, the purple and grey lines, respectively. There has been no statistically significant 
growth in the number of risk severity category RI events A (with collision), A (without collision) orB (near misses) 
overthe time period examined. The large increase in lower severity events between 2007 and 2008 is consistent 
with the previously noted findings of Chapman 11 . 

The Weather Condition data were sorted and searched to collect marginal probabilities for the correlation of 
various environmental conditions with RI events. Note that several sub-items have been merged together within 
each of the main categories of environmental conditions (Figure 7) to account for the inconsistencies in reporting of 
similar weather phenomena. Adverse lighting conditions may have the greatest overall and the most consistent 
contribution to severe runway events. These types of conditions were cited in about 25% of all the RI events with 
weather conditio ns noted, and to 90% Q, they were cited in 22% to 28% of the RI events; this is a relatively small 
band of uncertainty due to statisticalsignificance. A statistically significant correlation also exists between poor 
visibility and runway event risk, with more colhs ion events (25%) citing these conditions than RI e vents with ample 
time to avoid a collision (8%). Overall, snow (and other freezing conditions) is statistically less of a potential 
contributor to runway events (only 2% to 9% of RI events cite these conditions) than wind, rain, visibility and 
lighting. An unexplained significant in verse correlation exists between lightning and runway risk events. Possibly, 
the pace ofrunway operations is greatly slowed underthese circumstances due to enhanced vigilance of pilots and 
control personnel, leading to fewer RI events; however, this remains a topic for future investigation. 

The prevalence of RI events as a function of FAR category and risk severity is now examined. As noted 
previously, the three FARcategories of greatest interest for this study are 121 (commercial), 135 (air taxi) and 91 
(general aviation). Firs t , approximately 53% of all the reported RI events do not involve one of these three types of 
aircraft, or do not havean assumed risk level greater than zero. Table 1 shows values for each of the three FAR 
categories, in each of the risk severity categories, as a percentage (marginal probability) of all RI events reported. 
As observed within the tabular data, for those RI events of interest, by far the biggest total contribution is from 
general aviation (FAR Category 91) aircraft, which account for about 27% oftheRI events of interest. Commercial 
air carriers (FAR Category 121) account for almost 16% of the RI events of interest. Likewise, Category A risk 
severity events account for about 1% of all the RI events reported. 

A number of primary causes, contributing factors and interventions to RI events were discerned from the data. 
This analysis involves reading the narrative summary of each event and parsing from the narrative summary a 

5 


American Institute of Aeronautics and Astronautics 



sequence of events in order to establish the root cause of the RI event. This is a very time consuming manual 
process thatis somewhat subjective; eachofthe summaries were written by individualpeople at various places and 
times over the course of a decade. Although a common structure is usually employed, establishing the event 
sequence androotcauseof each RI event requires interpretation and judgment. An automated parsing scheme for a 
select list ofkey words and phrases, implemented in Microsoft Excel, was developed and applied by the author to 
the approximately 5000 RI events with narratives provided and many fewer (hundreds) were actually read 
completely. Again, the goal was to providereasonableguidanceabouttherelativepercentages of various kinds of 
events. 

The primary causes are summarized in Table 2. Among all the reported RI events, by far the most prevalent 
cause of RI events is pilot error, which accounts for about 72% of all the RI events examined to date, with an 
unauthorized person or vehicle accounting for about 19% of the RI events. Among the pilot errors, two contributing 
factors were readily identified as major contributors: accidental use of the wrong runway or taxiway (about 25% of 
pilot errors), and confusion abouttheextent of authority granted to the pilot at a specific time by the air / ground / 
local traffic control personnel (about 20% of the pilot errors). Surprisingly, only about 16% of the RI events 
examined included some foimof intervention, where a correctiveormitigative action taken. When an intervention 
or mitigation occurred, these actions were successful in reducing the RI event severity about 70% of the time. By 
far a “go-around” being issued to incoming planes was the most common form of intervention. 

A second and third round of data analysis for RI events was undertaken while this paper was in the approval 
process. The secondround examined 5020 RI events. These RI events were mostly overlapping with the original 
10459 records andcoveredthe period from 10/1/2007 through 12/5/2011, but with eventnarratives provided by the 
FAA. The data set revealed the following information: 

• 37.5% involved a single aircraft 

• 36.5% involved two aircraft 

• 14.3% involved a single vehicle 

• 6.9% involved a single aircraft and single vehicle 

• 3.7% involved a pedestrian only 

• 1.1% involved a single aircraft and a pedestrian 

The third round investigated 4038 RI events (a subset of the 5020 RI events noted above) which could easily be 
grouped into four sub categories, as shown in Figure 8: 1) a baseline group with no contributing weather factor and 
no mitigating actions (2325 RI events), 2) a group with weather as a contributing factor with no mitigating actions 
(884 RI events), 3) a group with mitigating actions but no weather as a contributing factor (609 RI events ), and 4) a 
group with both weather as a contributing factor and mitigating actions taken (220 RI events). The relative 
frequency of the various severity categories can be compared among these four groupings. As shown in Figure 9, 
and as expected, weather as acontributingfactorincreased the frequency of occurrence among category A through 
C severity RI events, relative to the baseline group. The probabilities of risk category A and B events each 
increasedby more than 30% in the presence of weather as a contributing factor. Mitigating actions reduced the 
frequency of occurrence of Category A and B severity RI events (in this data set the mitigating actions actually 
eliminated all category A and B severity RI events), but doubled the frequency of occurrence for category C events, 
compared to thebaseline group. The fourth group of RI events, with both weather as a contributing factor and 
mitigating actions taken also reduced the frequency of occurrence of Category A and B severity RI events and again 
doubled the frequency of occurrence for category C events, compared to the baseline group. 
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IV. Conclusions 


The number of runway incursion (RI) events is increasing as a function of calendar year in a statistically 
significant way. Using an assumed quantitative (numerical) risk value that is associated with each of the qualitative 
risk severity levels definedby theFAA,the actual risk associated with runway incursions is also increasing as a 
function ofcalendaryearin a statistically significant way. The air traffic volume (combined takeoffs and landings) 
for itinerant (air carrier, air taxi, general aviation and military) and local(civil and military) aircraft, and total airport 
operations, are decreasing as a function of calendar year in a statistically significant way. When considered together 
with the trends for risk events and assumed quantitative risk values, the normalized risk for RI is dramatically 
increasing. 

Many airports have a number of reported RI events well below the mean value of all airports, but numerous 
airports also have significantly highernumbers ofreportedRI events. Similar trends are observed for average risk 
and traffic normalized analyses. 


The only risk severity categories that haveexhibited a statistically significant growthover time are thos e for risk 
severity categories C and D, with assumed risk values of 2 and 1, respectively. There has been no statistically 
significant growth in the numberofriskseveritycategory RI events A (with collision), A (without collision) and B 
(near misses) over the time period examined. 

Among the environmental conditions examined, adverse lighting conditions appearto have the greatest overall, 
and the most consistent, contributionto RI events. A statistically significant correlation also exists between poor 
visibility and runway event risk, with more (percentage wise) collision / near miss events citing these conditions 
than RI events with ample time to avoid a colhs ion. Overall, snow (and other freezing conditions) is statistically less 
of a potential contributor to runway events than wind, rain, visibility and lighting. An unexplained significant 
inverse correlation exists between lightning and runway risk events. 

Among all thereportedRI events, the most prevalentcause of RI events is piloterror. Among the pilot errors, 
two sources were readily identified as major contributors: 1) accidental use of the wrong runway or taxiway, and 2) 
confusion about the extent of authority granted to the pilot at a specific time by the air / ground / local traffic control 
personnel. Only about 16% of the RI events examined noted some form of corrective or mitigative action taken; 
when taken theseactions were successful about 70% of the time and frequently resulted in a “go -around” being 
issued to incoming planes. 

It was shown that weather as a contributing factor increased the frequency of occurrence among category A 
throughC severity RI events, relative to the baseline group. Mitigating actions reduced the frequency of occurrence 
of Category A and B severity RI events but doubled the frequency of occurrence for category C events, compared to 
thebaselinegroup. The combination of weather as a contributing factor and mitigating actions taken also reduced 
the frequency of occurrence of Category A and B severity RI events and doubled the frequency of occurrence for 
category C events, compared to the baseline group. 
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Table 1. Correlation of FAR Aircraft Categories with RI Event Severity. 


FAR\Risk 

A 

B 

C 

D 

Total 

121 

0.33 

0.20 

7.73 

7.50 

15.76 

135 

0.12 

0.15 

1.51 

2.33 

4.12 

91 

0.56 

0.48 

8.78 

17.21 

27.03 

Total 

1.01 

0.83 

18.02 

27.04 



Table 2. Primary Causes of RI Events. 


Primary Cause 

Marginal Percentage 

Pilot Error 

72.07 

Unauthorized Person or Vehicle 

19.1 

Airport Ground Crew 

6.67 

Control Error 

2.16 
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Figure 1. Projected Number of RI Events as a function of calendar year. 
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Figure 2. Projected Runway Incursion Event Assumed Risk Sum by Year 
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Figure 3. Total Itinerant air traffic as a function of calendar year. 
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Figure 4. Reported RI Events by Airport, Mean Value RS Model. 
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Figure 5. Average Assumed Risk by Airport, Mean Value RS Model. 
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Figure 6. RI Event Count by Risk Severity Category and Year, Linear RS Models. 
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Figure 7. RI Event Correlations with Environmental Factors and Risk Severity Category. 
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Groups 
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Sum 
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2325 

884 

3209 
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609 
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2934 

1104 

4038 


Figure 8. Data Groups for Analysis of Contributing Factors 
and Mitigating Actions. 
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FACTORS 

A 

A+B 

A+B+C 

NONE 

BASELINE 

0.0082 

0.0129 

0.3247 

CONT 

0.0113 

0.0170 

0.3258 

MIT 

0.0000 

0.0000 

0.6591 

CONT+MIT 

0.0033 

0.0099 

0.6700 


Figure 9. Conditional probabilities related to weather as a contributing factor 
and the influence of mitigating actions. 
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